Emergency braking for autonomous vehicles

ABSTRACT

Aspects of the disclosure provide for generation of trajectories for a vehicle driving in an autonomous driving mode. For instance, information identifying a plurality of objects in the vehicle&#39;s environment and a confidence value for each of the objects is received. A set of constraints may be generated. That one or more processors are unable to solve for a trajectory given the set of constraints and an acceptable braking limit may be determined. A first constraint is identified as a constraint for which could not be solved and a first confidence value. That the vehicle should apply a maximum braking level is determined based on the identified first confidence value, a threshold, and the determination that the one or more processors are unable to solve for a trajectory. Based on the determination that the vehicle should apply the maximum braking level, the maximum braking level is applied.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.17/010,090, filed Sep. 2, 2020, which is a continuation of U.S.application Ser. No. 16/151,862, filed Oct. 4, 2018, now issued as U.S.Pat. No. 10,793,123, the entire disclosures of which are incorporated byreference herein.

BACKGROUND

Autonomous vehicles, such as vehicles that do not require a humandriver, can be used to aid in the transport of passengers or items fromone location to another. Such vehicles may operate in a fully autonomousmode where passengers may provide some initial input, such as a pickupor destination location, and the vehicle maneuvers itself to thatlocation by controlling various systems of the vehicles, includingacceleration and deceleration (braking) systems of the vehicle.

BRIEF SUMMARY

One aspect of the disclosure provides a method of generatingtrajectories for a vehicle driving in an autonomous driving mode. Themethod includes receiving, by one or more processors, from a perceptionsystem of the vehicle, information identifying a plurality of objects inan environment of the vehicle and a confidence value for each object ofthe plurality of objects being a real object; generating, by the one ormore processors, a set of constraints, a first constraint of the set ofconstraints being generated based on a first object of the objects ofthe plurality of objects; determining, by the one or more processors,that the one or more processors are unable to solve for a trajectorygiven the set of constraints and an acceptable braking limit;identifying, by the one or more processors, the first constraint as aconstraint for which the one or more processors were unable to solve fora trajectory and a first confidence value, the first constraintcorresponding to the first object that is associated with the firstconfidence value; determining, by the one or more processors, that thevehicle should apply a maximum braking level greater than the acceptablebraking limit based on the identified first confidence value, athreshold, and the determination that the one or more processors areunable to solve for a trajectory; and based on the determination thatthe vehicle should apply the maximum braking level, applying, by the oneor more processors, the maximum braking level to control the vehicle inthe autonomous driving mode and in order to slow the vehicle down.

In one example, the method also includes for each given object of theplurality of objects, determining a priority level based on theconfidence value and a type of the given object; comparing thedetermined priority levels to a threshold value; and based on thecomparison, flagging at least the first constraint to indicate that thefirst object allows for the maximum braking level. In this example,determining that the vehicle should apply a maximum braking level isfurther based on the flagged first constraint. In another example, themethod also includes for each given object of the plurality of objects,determining a priority level based on the confidence value and a type ofthe given object; comparing the determined priority levels to athreshold value; and based on the comparison, flagging at least thefirst object to indicate that the first object allows for the maximumbraking level. In this example, determining that the vehicle shouldapply a maximum braking level is further based on the flagged firstobject. In another example, the first confidence value corresponds to anumber of iterations of a LIDAR sensor of the perception system in whichthe one of the plurality of objects has been observed. In anotherexample, the method also includes receiving for the first object, a typeof the first object; and determining a priority level based on the firstconfidence value and the type of object, and wherein determining thatthe vehicle should apply a maximum braking level is further based on thepriority level. In this example, determining that the vehicle shouldapply a maximum braking level is further based on a comparison betweenthe priority level and the threshold. In addition or alternatively, themethod also includes determining that the priority level meets thethreshold, and wherein determining that the vehicle should apply themaximum braking level is further based on the determination that thepriority level meets the threshold. In another example, determining thatthe vehicle should apply the maximum braking level is further based on acurrent speed of the vehicle. In another example, determining that thevehicle should apply the maximum braking level is further based on adetected speed of one or more of the plurality of objects. In anotherexample, determining that the vehicle should apply the maximum brakinglevel is further based on a type of road on which the vehicle iscurrently driving. In another example, determining that the vehicleshould apply the maximum braking level is further based on whether thevehicle is currently driving on a highway. In another example, themaximum braking level corresponds to a maximum braking limit for brakesof the vehicle. In another example, determining that the vehicle shouldapply the maximum braking level is further based on a speed limit of aroad on which the vehicle is traveling. In another example, determiningthat the vehicle should apply the maximum braking level is further basedon a speed of a second vehicle in the environment of the vehicle. Inanother example, applying the maximum braking level is further based onwhether a second vehicle is behind the vehicle. In this example,applying the maximum braking level is further based on whether thesecond vehicle meets a predetermined threshold with respect to a currentlocation of the vehicle. In another example, determining that thevehicle should apply the maximum braking level is further based on acurrent driving situation for the vehicle, wherein the current drivingsituation relates to an environment in which the vehicle is currentlydriving.

Another aspect of the disclosure provides a system for generatingtrajectories for a vehicle driving in an autonomous driving mode. Thesystem includes one or more processors configured to receive from aperception system of the vehicle, information identifying a plurality ofobjects in an environment of the vehicle and a confidence value for eachobject of the plurality of objects being a real object; generate a setof constraints, a first constraint of the set of constraints beinggenerated based on a first object of the objects of the plurality ofobjects; determine that the one or more processors are unable to solvefor a trajectory given the set of constraints and an acceptable brakinglimit; identify the first constraint as a constraint for which the oneor more processors were unable to solve and a first confidence value,the first constraint corresponding to the first object that isassociated with the first confidence value; determine that the vehicleshould apply a maximum braking level greater than the acceptable brakinglimit based on the identified first confidence value, a threshold, andthe determination that the one or more processors are unable to solvefor a trajectory; and based on the determination that the vehicle shouldapply the maximum braking level, apply the maximum braking level tocontrol the vehicle in the autonomous driving mode and in order to slowthe vehicle down. In one example, the system also includes the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of an example vehicle in accordance withaspects of the disclosure.

FIG. 2 is an example representation of map information in accordancewith aspects of the disclosure.

FIG. 3 is an example external view of a vehicle in accordance withaspects of the disclosure.

FIG. 4 is a view of a section of roadway in accordance with aspects ofthe disclosure.

FIG. 5 is a view of a section of roadway and data in accordance withaspects of the disclosure.

FIG. 6 is another view of a section of roadway and data in accordancewith aspects of the disclosure.

FIG. 7 is another view of a section of roadway and data in accordancewith aspects of the disclosure.

FIG. 8 is another view of a section of roadway and data in accordancewith aspects of the disclosure.

FIG. 9 is another view of a section of roadway and data in accordancewith aspects of the disclosure.

FIG. 10 is flow diagram in accordance with aspects of the disclosure.

DETAILED DESCRIPTION Overview

The technology relates to allowing a vehicle to initiate emergencybraking when the confidence an object is in the path of the vehicle ishigh. In this regard, a vehicle, such as an autonomous vehicle, mayfollow a projected path at a certain speed and/or acceleration. When anobject is detected in the vehicle's projected path, the vehicle may berequired to make an emergency maneuver, such as sharply turning orbraking hard to avoid colliding with the object. As such, objects whichhave low confidence values, indicating that there is a low certainty ofthe objects actually being present, may result in unnecessary hardbraking being performed by the autonomous vehicle. However, these hardbraking may increase the risk of other vehicles colliding with theautonomous vehicle (for instance, a tailgater may rear-end the vehicle)and decrease rider comfort. To address this, hard braking may only beenabled when the detected objects have high confidence values.

As the vehicle drives around, the vehicle's perception system may detectand identify objects and their characteristics. Each detected object maybe associated with a confidence value identifying how confident theperception system is that the object is “real.” The more the object isperceived by the sensors of the perception system, the greater theconfidence value that the object is a real, correctly identified object,with more reliable estimates of position, orientation and speed. Theconfidence values and type for each identified object may be used todetermine a priority level for each object.

The vehicle's computing devices may also determine whether the prioritylevel for each object meets a threshold. If so, for instance because thepriority is greater than or equal to the threshold, the constraint forthat object and/or the object may be flagged or otherwise marked asallowing for an emergency braking or maximum braking level.

The vehicle's computing devices may generate a set of constraints basedon the objects. For instance, the objects and their characteristics,such as location, speed, heading, orientation, predicted behaviors, etc.may be used to generate a set of constraints identifying locations atcertain times at which the vehicle should not be in order to avoidcollision.

The set of constraints may be used to attempt to generate trajectoriesthat allow the vehicle to maintain safe and comfortable travelingconditions. For instance, the set of constraints may be used to generatea trajectory for the vehicle by solving for a trajectory that satisfiesall of the constraints while allowing the vehicle to brake at no greaterthan an acceptable braking level. In certain circumstances, thevehicle's computing devices may not be able to solve for all of theconstraints. In such situations, the vehicle's computing devices mayapply the brakes of the vehicle at the maximum braking level and, insome cases, also employ a parking brake if available. As such, thevehicle's computing devices may automatically apply the maximum brakinglevel without specifically attempting to solve for the set ofconstraints.

However, as noted above, higher levels of braking limit may increaserisk of other vehicles colliding with the autonomous vehicle anddecrease rider comfort. As such, before applying the maximum brakinglevel, the vehicle's computing devices may identify a constraint forwhich the vehicle's computing devices are not able to solve for atrajectory and determine whether the constraint and/or the correspondingobject for that constraint has been flagged to allow for maximum brakinglevel.

The features described herein may allow the vehicle's computing devicesto use emergency braking in a safe and reliable way. By tying thisbraking to the confidence value that an object detected by theperception system exists as well as the type of that object, thisreduces the likelihood of unnecessary, and in some cases, dangerousbraking in inappropriate circumstances. This, in turn, increases thesafety and comfort of passengers within the vehicle.

Example Systems

As shown in FIG. 1, a vehicle 100 in accordance with one aspect of thedisclosure includes various components. While certain aspects of thedisclosure are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, busses, recreational vehicles,etc. The vehicle may have one or more computing devices, such ascomputing device 110 containing one or more processors 120, memory 130and other components typically present in general purpose computingdevices.

The memory 130 stores information accessible by the one or moreprocessors 120, including instructions 132 and data 134 that may beexecuted or otherwise used by the processor 120. The memory 130 may beof any type capable of storing information accessible by the processor,including a computing device-readable medium, or other medium thatstores data that may be read with the aid of an electronic device, suchas a hard-drive, memory card, ROM, RAM, DVD or other optical disks, aswell as other write-capable and read-only memories. Systems and methodsmay include different combinations of the foregoing, whereby differentportions of the instructions and data are stored on different types ofmedia.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computingdevice code on the computing device-readable medium. In that regard, theterms “instructions” and “programs” may be used interchangeably herein.The instructions may be stored in object code format for directprocessing by the processor, or in any other computing device languageincluding scripts or collections of independent source code modules thatare interpreted on demand or compiled in advance. Functions, methods androutines of the instructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. As an example, data 134 of memory130 may store predefined scenarios. A given scenario may identify a setof scenario requirements including a type of object, a range oflocations of the object relative to the vehicle, as well as otherfactors such as whether the autonomous vehicle is able to maneuveraround the object, whether the object is using a turn signal, thecondition of a traffic light relevant to the current location of theobject, whether the object is approaching a stop sign, etc. Therequirements may include discrete values, such as “right turn signal ison” or “in a right turn only lane”, or ranges of values such as “havingan heading that is oriented at an angle that is 30 to 60 degrees offsetfrom a current path of vehicle 100.” In some examples, the predeterminedscenarios may include similar information for multiple objects.

The one or more processor 120 may be any conventional processors, suchas commercially available CPUs. Alternatively, the one or moreprocessors may be a dedicated device such as an ASIC or otherhardware-based processor. Although FIG. 1 functionally illustrates theprocessor, memory, and other elements of computing device 110 as beingwithin the same block, it will be understood by those of ordinary skillin the art that the processor, computing device, or memory may actuallyinclude multiple processors, computing devices, or memories that may ormay not be stored within the same physical housing. As an example,internal electronic display 152 may be controlled by a dedicatedcomputing device having its own processor or central processing unit(CPU), memory, etc. which may interface with the computing device 110via a high-bandwidth or other network connection. In some examples, thiscomputing device may be a user interface computing device which cancommunicate with a user's client device. Similarly, the memory may be ahard drive or other storage media located in a housing different fromthat of computing device 110. Accordingly, references to a processor orcomputing device will be understood to include references to acollection of processors or computing devices or memories that may ormay not operate in parallel.

Computing device 110 may all of the components normally used inconnection with a computing device such as the processor and memorydescribed above as well as a user input 150 (e.g., a mouse, keyboard,touch screen and/or microphone) and various electronic displays (e.g., amonitor having a screen or any other electrical device that is operableto display information). In this example, the vehicle includes aninternal electronic display 152 as well as one or more speakers 154 toprovide information or audio visual experiences. In this regard,internal electronic display 152 may be located within a cabin of vehicle100 and may be used by computing device 110 to provide information topassengers within the vehicle 100. In addition to internal speakers, theone or more speakers 154 may include external speakers that are arrangedat various locations on the vehicle in order to provide audiblenotifications to objects external to the vehicle 100.

In one example, computing devices 110 may be control computing devicesof an autonomous driving computing system or incorporated into vehicle100. The autonomous driving computing system may capable ofcommunicating with various components of the vehicle in order to controlthe movement of vehicle 100 according to the autonomous control softwareof memory 130 as discussed further below. For example, returning to FIG.1, computing devices 110 may be in communication with various systems ofvehicle 100, such as deceleration system 160 (for instance, includingthe features connected or otherwise attached to the brake pedal of thevehicle), acceleration system 162 (for instance, including the featuresconnected or otherwise attached to the accelerator pedal of thevehicle), steering system 164, signaling system 166, routing system 168,positioning system 170, perception system 172, and power system 174(i.e. the vehicle's engine or motor) in order to control the movement,speed, etc. of vehicle 100 in accordance with the instructions 132 ofmemory 130. For instance, the computing devices 110 may generatetrajectories for the vehicle 100 to follow and control the vehicleaccording to the trajectory by sending commands to the decelerationsystem 160 (to control braking power and accordingly, deceleration) aswell as acceleration system 162 (to control acceleration and changes inacceleration). Again, although these systems are shown as external tocomputing devices 110, in actuality, these systems may also beincorporated into computing devices 110, again as an autonomous drivingcomputing system for controlling vehicle 100.

The computing device 110 may control the direction and speed of thevehicle by controlling various components. By way of example, computingdevice 110 may navigate the vehicle to a destination location completelyautonomously using data from the map information and navigation system168. Computing devices 110 may use the positioning system 170 todetermine the vehicle's location and perception system 172 to detect andrespond to objects when needed to reach the location safely. In order todo so, computing devices 110 may cause the vehicle to accelerate (e.g.,by increasing fuel or other energy provided to the engine byacceleration system 162), decelerate (e.g., by decreasing the fuelsupplied to the engine, changing gears, and/or by applying brakes bydeceleration system 160), change direction (e.g., by turning the frontor rear wheels of vehicle 100 by steering system 164), and signal suchchanges (e.g., by lighting turn signals of signaling system 166). Thus,the acceleration system 162 and deceleration system 160 may be a part ofa drivetrain that includes various components between an engine of thevehicle and the wheels of the vehicle. Again, by controlling thesesystems, computing devices 110 may also control the drivetrain of thevehicle in order to maneuver the vehicle autonomously.

As an example, computing device 110 may interact with decelerationsystem 160 and acceleration system 162 in order to control the speed ofthe vehicle. Similarly, steering system 164 may be used by computingdevice 110 in order to control the direction of vehicle 100. Forexample, if vehicle 100 configured for use on a road, such as a car ortruck, the steering system may include components to control the angleof wheels to turn the vehicle. Signaling system 166 may be used bycomputing device 110 in order to signal the vehicle's intent to otherdrivers or vehicles, for example, by lighting turn signals or brakelights when needed.

Navigation system 168 may be used by computing device 110 in order todetermine and follow a route to a location. In this regard, thenavigation system 168 and/or data 134 may store map information, e.g.,highly detailed maps that computing devices 110 can use to navigate orcontrol the vehicle. As an example, these maps may identify the shapeand elevation of roadways, lane markers, intersections, crosswalks,speed limits, traffic signal lights, buildings, signs, real time orhistorical traffic information, vegetation, or other such objects andinformation. The lane markers may include features such as solid orbroken double or single lane lines, solid or broken lane lines,reflectors, etc. A given lane may be associated with left and right lanelines or other lane markers that define the boundary of the lane. Thus,most lanes may be bounded by a left edge of one lane line and a rightedge of another lane line. As noted above, the map information may storeknown traffic or congestion information and/or and transit schedules(train, bus, etc.) from a particular pickup location at similar times inthe past. This information may even be updated in real time byinformation received by the computing devices 110.

FIG. 2 is an example of map information 200 for a section of roadwayincluding intersection 220. In this example, map information 200 depictsa portion of the map information that includes information identifyingthe shape, location, and other characteristics of lane markers or lanelines 210, 212, 214, stop signs 240, 242, 244, 246, as well as stoplines 250, 252, 254, 256. The lane lines may also define various lanes260, 261, 264, 265, implied by the width of a road, such as lanes 262,263, 266, 267, and/or lanes 260-267 may also be explicitly identified inthe map information 200. In this example, map information 200 alsoidentifies bicycle lanes 280, 282. In addition to these features, themap information may also include information that identifies thedirection of traffic and speed limits for each lane as well asinformation that allows the computing devices 110 to determine whetherthe vehicle has the right of way to complete a particular maneuver (i.e.complete a turn or cross a lane of traffic or intersection), as well asother features such as curbs, buildings, waterways, vegetation, signs,etc.

Although the detailed map information is depicted herein as animage-based map, the map information need not be entirely image based(for example, raster). For example, the detailed map information mayinclude one or more roadgraphs or graph networks of information such asroads, lanes, intersections, and the connections between these features.Each feature may be stored as graph data and may be associated withinformation such as a geographic location and whether or not it islinked to other related features, for example, a stop sign may be linkedto a road and an intersection, etc. In some examples, the associateddata may include grid-based indices of a roadgraph to allow forefficient lookup of certain roadgraph features.

The perception system 172 also includes one or more components fordetecting objects external to the vehicle such as other vehicles,obstacles in the roadway, traffic signals, signs, trees, etc. Forexample, the perception system 172 may include one or more LIDARsensors, sonar devices, radar units, cameras and/or any other detectiondevices that record data which may be processed by computing devices110. The sensors of the perception system may detect objects and theircharacteristics such as location, orientation, size, shape, type (forinstance, vehicle, pedestrian, bicyclist, etc.), heading, and speed ofmovement, etc. The raw data from the sensors and/or the aforementionedcharacteristics can be quantified or arranged into a descriptivefunction, vector, and or bounding box and sent for further processing tothe computing devices 110 periodically and continuously as it isgenerated by the perception system 172. As discussed in further detailbelow, computing devices 110 may use the positioning system 170 todetermine the vehicle's location and perception system 172 to detect andrespond to objects when needed to reach the location safely.

For instance, FIG. 3 is an example external view of vehicle 100. In thisexample, roof-top housing 310 and dome housing 312 may include a LIDARsensor as well as various cameras and radar units. In addition, housing320 located at the front end of vehicle 100 and housings 330, 332 on thedriver's and passenger's sides of the vehicle may each store a LIDARsensor. For example, housing 330 is located in front of driver door 350.Vehicle 100 also includes housings 340, 342 for radar units and/orcameras also located on the roof of vehicle 100. Additional radar unitsand cameras (not shown) may be located at the front and rear ends ofvehicle 100 and/or on other positions along the roof or roof-top housing310. Vehicle 100 also includes many features of a typical passengervehicle such as doors 350, 352, wheels 360, 362, etc.

Once a nearby object is detected, computing devices 110 and/orperception system 172 may determine the object's type, for example, atraffic cone, pedestrian, a vehicle (such as a passenger car, truck,bus, etc.), bicycle, etc. Objects may be identified by various modelswhich may consider various characteristics of the detected objects, suchas the size of an object, the speed of the object (bicycles do not tendto go faster than 40 miles per hour or slower than 0.1 miles per hour),the heat coming from the bicycle (bicycles tend to have rider that emitheat from their bodies), etc. In addition, the object may be classifiedbased on specific attributes of the object, such as informationcontained on a license plate, bumper sticker, or logos that appear onthe vehicle.

In one example, computing devices 110 may be operable to predict anothervehicle's future movement based solely on the other vehicle's instantdirection, acceleration/deceleration and velocity, e.g., that the othervehicle's current direction and movement will continue. However, memory130 may also store behavior models that provides the probability of oneor more actions being taken a detected object. To increase theusefulness of these behavior models, each behavior model may beassociated with a specific type of object. For instance, one type ofbehavior model may be used for objects identified as pedestrians,another type of behavior model may be used for objects identified asvehicles, another type of behavior may be used for objects identified asbicycles or bicyclists, etc. The behavior models may be used by thecomputing devices 110 in order to predict a detected object futuremovement by analyzing data relating to the other vehicle's currentsurroundings and determining how that other object will likely respondto those surroundings. In this regard, the behavior models may functionfrom an object-centric, view of the object's environment, in that thesystem determines what the other objects are perceiving in order tobetter predict how those objects will behave. In this regard, in atleast some instances, the behavior models may also indicate whether thepredicted behavior for an object is responsive to a particular otherobject including the vehicle 100.

Example Methods

In addition to the operations described above and illustrated in thefigures, various operations will now be described. It should beunderstood that the following operations do not have to be performed inthe precise order described below. Rather, various steps can be handledin a different order or simultaneously, and steps may also be added oromitted.

As noted above, the computing devices 110 may use the routing system todetermine a route to a destination. The computing devices 110 may thenperiodically generate trajectories, for instance every 0.1 times persecond or more or less, for the vehicle to follow for some time anddistance into the future in order to follow the route to thedestination. As noted above, these trajectories are generated as“desired paths” in order to avoid obstacles, obey laws and generallydrive safely and effectively. Each trajectory may define variousrequirements for the vehicle's acceleration, speed, and position atdifferent times along the trajectory. In this regard, the trajectory mayinclude both geometry and speed/acceleration components. The computingdevices 110 may operate to manage steering, powertrain and braking inorder to cause the vehicle to follow the trajectory. The computingdevices 110 may receive feedback from the vehicles sensors includingsensors of and/or the perception system 172 and the positioning system170. This feedback may include information such as the currentacceleration, speed, and position of the vehicle as well as thosecharacteristics of objects detected in the vehicle's environment.

FIG. 4 is an example 400 bird's eye view of a section of roadwaycorresponding to map information 200. In this example, the shapelocation and other characteristics of intersection 420, lane lines 410,412, 414, stop signs 440, 442, 444, 446, stop lines 450, 452, 454, 456,lanes 460-467, and bicycle lanes 480, 482, may generally correspond tothe shape location and other characteristics of intersection 220, lanelines 210, 212, 214, stop signs 240, 242, 244, 246, stop lines 250, 252,254, 256, lanes 260-268, and bicycle lanes 280, 282 respectively. Inthis example, vehicle 100 is driving in lane 460 towards intersection420. In addition, another vehicle 490 is also driving in lane 460towards intersection 420, and a bicyclist 492 is riding in bike lane 480towards intersection 420 as well.

As the vehicle drives around, the perception system 172 and/or computingdevices 110 may detect and identify objects and their characteristics.For instance, as shown in example 500 of FIG. 5, the perception system172 may detect various objects in the vehicle's environment. In additionto the features of the map information 200, the perception system 172may also detect and identify objects such as objects represented bybounding boxes 590, 592, and 594. In this example, the bounding boxesrepresent an estimated size, shape, orientation and location of thedetected objects and may “bound” the data points, such as LIDAR sensordata points or camera image data points, relative to the example 600.

Each detected object may be associated with a confidence valueidentifying how confident the perception system is that the object is“real.” These confidence values may be determined by the perceptionsystem 172 and/or computing devices 110. For instance, if an object or apoint cloud is only perceived by a single sensor, such as a LIDARsensor, 1 or 2 times (1 or 2 sensor iterations), it is possible that theobject is not a real object, but simply noise, reflections, or exhaustvapor from another vehicle. Similarly, some objects, such as smallvegetation or debris, may appear to be moving and may be detected asanother road user, but may simply be swaying in the wind. Thus, theconfidence value would be very low. For instance, bounding box 594 maybe associated with a very low confidence value. As such, turning toexample 600 of FIG. 6, bounding box 594 may represent data points for anobject that has only been perceived during 1 or 2 sensor iterations. Assuch, this bounding box is associated with a very low confidence of 0.1on a scale of 0 to 1, where zero is low confidence and 1 is highconfidence. Of course, other scales and values may be used. Moreover,the fewer the number of iterations of the LIDAR sensor that the objectis identified, the more difficult it is to determine the object'scharacteristics such as speed, type, and heading as well as to predictthe object's behavior (for instance, using behavior models as discussedabove). As such, estimates made by the perception system 172 and/orcomputing devices 110 of the bounding box 594's speed, type, heading,and behavior may be very inaccurate. In this regard, bounding box 594may be estimated to be a small vehicle, but in reality may simply beexhaust from vehicle 490 and/or other road debris such as leaves, trash,etc.

The more the object is perceived, for instance, the longer an object istracked or the more sensor iterations in which the object is detected bythe sensors of the perception system, the greater the confidence valuethat the object is a real, correctly identified object, with morereliable estimates of position, orientation and speed. In this regard,higher confidence value objects may be perceived over several, forinstance at least 3-5 or more iterations of the perception system. As anexample, bounding boxes 590 and 592, representing vehicle 490 andbicyclist 492, respectively, are each associated with a relatively highconfidence value, here 0.9 on the aforementioned scale of 0 to 1, giventhe fact that these objects have been observed over the course of manydifferent sensor iterations. In addition, the characteristics of theseobjects, such as speed, type, heading as well as behavior predictionsmay also be more accurate. In this regard, bounding box 590 may beidentified as a vehicle by perception system 172 and/or computingdevices 110, and bounding box 592 may be identified as a bicyclist byperception system 172 and/or computing devices 110.

The confidence values and identified type for each identified object maybe used by the computing devices 110 to determine a priority level forthose objects. For instance, objects for which the vehicle is morelikely to need to slow down, such as vehicles, bicyclists andpedestrians may be assigned a higher priority than objects such astraffic cones, vegetation, or trash. In addition, objects having higherconfidence values may be assigned higher priority. For instance,vegetation having a higher confidence value would have a higher prioritythan vegetation with a lower confidence value. moreover, rather thanbeing a ranked priority level, the priority level may simply be on afixed scale, such as 0 to 1 where 0 is low priority and 1 is highpriority, such that different objects may have the same priority levelif such objects would have the same type and confidence value. Again,other scales and values may be used. Turning to example 700 of FIG. 7,given their identified types and confidence values, bounding boxes 590and 592 may be relatively high, or 0.8 and 0.9, respectively, on theaforementioned scale of 0 to 1. However, although bounding box 594 hasbeen identified as a vehicle, because the confidence value is relativelylow, 0.1, the priority of the bounding box 594, is also relatively low,or 0.2 on a scale of 0 to 1.

The computing devices 110 may generate a set of constraints. Forinstance, the objects and their characteristics, such as location,speed, heading, orientation, projected behaviors, etc. may be used togenerate a set of constraints identifying locations at certain times atwhich the vehicle should not be in order to avoid collision. As anexample, the constraints may include avoiding or not coming within somepredetermined distances of locations on predicted behaviors ortrajectories of detected objects. For instance, turning to example 900of FIG. 9, arrows 990, 992, and 994 represent predicted trajectories foreach of bounding boxes 590, 592, and 594, respectively which may bepredicted based on the estimated characteristics of these objects andthe map information as discussed above.

The vehicle's computing devices may also determine whether the prioritylevel for each object meets a threshold. If so, for instance because thepriority is greater than or equal to the threshold, the constraint forthat object and/or the object may be flagged or otherwise marked asallowing for a maximum braking level. In this regard, if the priority istoo low, the constraint for that object may not allow for the maximumbraking level. As such, only higher priority level objects of certaintypes with high confidence values would be flagged to allow foremergency braking. For instance, turning to example 800 of FIG. 8, ifthe threshold is 0.6, both bounding boxes 590 and 592, which havepriority levels that are higher than 0.6, as shown as flagged to allowfor emergency braking. Similarly, lower priority level objects of othertypes with low confidence values would not be flagged to allow foremergency braking. In this regard, bounding box 594 is shown as notbeing flagged to allow for emergency braking. In addition oralternatively, constraints may be flagged as not allowing for maximumbraking level.

The set of constraints may be used to attempt to generate trajectoriesthat allow the vehicle to maintain safe and comfortable travelingconditions. For instance, the set of constraints may be used to generatea trajectory for the vehicle by solving for a trajectory that satisfiesall of the constraints while allowing the vehicle to brake at no greaterthan an acceptable braking level. The trajectories may be determined asa single problem (solving for geometry and speed at once) or a two-partproblem (solving for geometry and thereafter speed).

The maximum braking level is greater than the acceptable braking level.For instance, the acceptable braking level may define a maximumdeceleration and/or a maximum amount of jerk (derivative ofdeceleration) for the vehicle. These maximums for the acceptable brakinglevel may be hand tuned in order to be comfortable to any passenger inthe vehicle and reduce the likelihood of the vehicle being rear-ended byother vehicles. The maximum braking level may thus be a maximum amountof braking, maximum amount of deceleration and/or a maximum amount ofjerk of which the vehicle is capable in order to slow the vehicle downas quickly as possible.

In certain circumstances, the vehicle's computing devices may not beable to solve for all of the constraints. For instance, the vehicle'scomputing devices may not be able to find a speed plan for a specificgeometry that satisfies all of the constraints of a set of constraints.As an example, computing devices 110 may not be able to identify a speedplan which avoids coming too close to the constraints associated withthe object represented by bounding box 592 corresponding to bicyclist492. As another example, computing devices 110 may not be able toidentify a speed plan which avoids coming too close to the constraintsassociated with the object represented by bounding box 594 correspondingto exhaust and/or other road debris. In such situations, the vehicle'scomputing devices may apply the brakes of the vehicle at the maximumbraking level and, in some cases, also employ a parking brake ifavailable. As such, the vehicle's computing devices may automaticallyapply the maximum braking level without specifically attempting to solvefor the set of constraints.

However, as noted above, higher levels of braking limit may increaserisk of other vehicles colliding with the autonomous vehicle anddecrease rider comfort. As such, before applying the maximum brakinglevel, the vehicle's computing devices may identify a constraint forwhich the vehicle's computing devices are not able to solve for atrajectory and determine whether the constraint and/or the correspondingobject for that constraint has been flagged to allow for maximum brakinglevel or really, whether the priority for the object associated with theidentified constraint meets a threshold value that would allow formaximum braking level. If so, the vehicle's computing devices may simplyautomatically apply the maximum braking level without specificallyattempting to solve for the set of constraints. For instance, ifcomputing devices 110 may not be able to identify a speed plan whichavoids coming too close to the constraints associated with the objectrepresented by bounding box 592, because this bounding box has alreadybeen flagged and/or is associated with a priority level which meets thethreshold, the computing devices may automatically apply the maximumbraking level. As another example, if computing devices 110 may not beable to identify a speed plan which avoids coming too close to theconstraints associated with the object represented by bounding box 594while avoiding coming too close to bounding box 592, because boundingbox 594 has not been flagged and/or is not associated with a prioritylevel which meets the threshold, the computing devices would notautomatically apply the maximum braking level.

At the same time, it may be appropriate to apply the maximum brakinglevel in situations where the vehicle is very unlikely to cause suchhigh risk events. In other words, not all situations may be suitable foremergency braking. For example, it may not be appropriate to increasethe acceptable braking limit on a highway where the speed of trafficflow speed is relatively high as this may cause a high risk to thevehicle, passengers as well as other vehicles. In this regard, themaximum braking level may be allowed only when the vehicle's speed andthe speed of other nearby vehicles are relatively low such that even ifa collision were to happen, the collision would be of very low severity.As another example, before applying the maximum braking level, thecomputing devices 110 may first confirm that there are no tailgatingvehicles behind vehicle 100, such that there are no vehicles within apredetermined threshold, measured in distances (e.g. meters) or time(i.e. seconds) or a combination of both, in a lane of and behind thecurrent location of the vehicle 100 such that the risk of a rearcollision is low.

FIG. 10 is a flow diagram 1000 that may be performed by one or moreprocessors such as one or more processors 120 of computing device 110 inorder to perform a method of generating trajectories for a vehicledriving in an autonomous driving mode. In this example, at block 1010,information identifying a plurality of objects in the vehicle'senvironment and a confidence value for each object of the plurality ofobjects being a real object is received from a perception system of thevehicle. At block 1020, a set of constraints is generated. A firstconstraint of the set of constraints is generated based on a firstobject of the objects of the plurality of objects. At block 1030, thatthe one or more processors are unable to solve for a trajectory giventhe set of constraints and an acceptable braking limit is determined. Atblock 1040, a first constraint of the set of constraints for which theone or more processors were unable to solve is identified. The firstconstraint corresponds to a first object of the plurality of objectsassociated with a first confidence value. At block 1050, that thevehicle should apply a maximum braking level greater than the acceptablebraking limit based on the first confidence, a threshold, and thedetermination that the one or more processors are unable to solve for atrajectory is determined. At block 1060, based on the determination thatthe vehicle should apply the maximum braking level, the maximum brakinglevel is applied to control the vehicle in the autonomous driving modeand in order to slow the vehicle down.

The features described herein may allow the vehicle's computing devicesto use emergency braking in a safe and reliable way. By tying thisbraking to the confidence value that an object detected by theperception system exists as well as the type of that object, thisreduces the likelihood of unnecessary, and in some cases, dangerousbraking in inappropriate circumstances. For instance, if a child were tostep in front of the vehicle onto the road, because even where theconfidence value for the perception of that child were low, becausechildren may have high priority level, the vehicle would automaticallybe able to use emergency braking. At the same time, if the perceptionsystem identifies an object with a low confidence value that emergesfrom behind a tree that is later not confirmed or observed again, thepriority would be very low and thus, the vehicle would notinappropriately use the emergency braking. This, in turn, increases thesafety and comfort of passengers within the vehicle.

Unless otherwise stated, the foregoing alternative examples are notmutually exclusive, but may be implemented in various combinations toachieve unique advantages. As these and other variations andcombinations of the features discussed above can be utilized withoutdeparting from the subject matter defined by the claims, the foregoingdescription of the embodiments should be taken by way of illustrationrather than by way of limitation of the subject matter defined by theclaims. In addition, the provision of the examples described herein, aswell as clauses phrased as “such as,” “including” and the like, shouldnot be interpreted as limiting the subject matter of the claims to thespecific examples; rather, the examples are intended to illustrate onlyone of many possible embodiments. Further, the same reference numbers indifferent drawings can identify the same or similar elements.

1. A method of generating trajectories for a vehicle driving in anautonomous driving mode, the method comprising: receiving, by one ormore processors, from a perception system of the vehicle, informationidentifying a plurality of objects in an environment of the vehicle anda confidence value for each object of the plurality of objects being areal object; determining, by the one or more processors, that thevehicle should apply a braking level greater than the acceptable brakinglimit based on the confidence value for the first object; and based onthe determination that the vehicle should apply the braking level,applying, by the one or more processors, the braking level to controlthe vehicle in the autonomous driving mode and in order to slow thevehicle down.
 2. The method of claim 1, further comprising: for eachgiven object of the plurality of objects, determining a priority levelbased on the confidence value and a type of the given object; comparingthe determined priority levels to a threshold value; and based on thecomparison, determining that the first object allows for the brakinglevel.
 3. The method of claim 1, wherein the breaking level is a maximumbraking limit for brakes of the vehicle.
 4. The method of claim 1,wherein the first confidence value corresponds to a number of iterationsof a LIDAR sensor of the perception system in which the one of theplurality of objects has been observed.
 5. The method of claim 1,further comprising: receiving for the first object, a type of the firstobject; and determining a priority level based on the first confidencevalue and the type of object, and wherein determining that the vehicleshould apply the braking level is further based on the priority level.6. The method of claim 7, determining that the vehicle should apply thebraking level is further based on a comparison between the prioritylevel and the threshold.
 7. The method of claim 7, further comprising,determining that the priority level meets the threshold, and whereindetermining that the vehicle should apply the braking level is furtherbased on the determination that the priority level meets the threshold.8. The method of claim 1, wherein determining that the vehicle shouldapply the braking level is further based on a current speed of thevehicle.
 9. The method of claim 1, wherein determining that the vehicleshould apply the braking level is further based on a detected speed ofone or more of the plurality of objects.
 10. The method of claim 11,wherein determining that the vehicle should apply the braking level isfurther based on a current speed of the vehicle.
 11. The method of claim1, wherein determining that the vehicle should apply the braking levelis further based on a type of road on which the vehicle is currentlydriving.
 12. The method of claim 1, wherein determining that the vehicleshould apply the braking level is further based on whether the vehicleis currently driving on a highway.
 13. The method of claim 1, whereindetermining that the vehicle should apply the braking level is furtherbased on a speed limit of a road on which the vehicle is traveling. 14.The method of claim 1, wherein determining that the vehicle should applythe braking level is further based on a speed of a second vehicle in theenvironment of the vehicle.
 15. The method of claim 1, wherein applyingthe braking level is further based on whether a second vehicle is behindthe vehicle.
 16. The method of claim 17, wherein applying the brakinglevel is further based on whether the second vehicle meets apredetermined threshold with respect to a current location of thevehicle.
 17. The method of claim 1, wherein determining that the vehicleshould apply the braking level is further based on a current drivingsituation for the vehicle, wherein the current driving situation relatesto an environment in which the vehicle is currently driving.
 18. Asystem for generating trajectories for a vehicle driving in anautonomous driving mode, the system comprising: one or more processorsconfigured to: receive from a perception system of the vehicle,information identifying a plurality of objects in an environment of thevehicle and a confidence value for each object of the plurality ofobjects being a real object; determine that the vehicle should apply abraking level greater than the acceptable braking limit based on theconfidence value for the object; and based on the determination that thevehicle should apply the braking level, apply the braking level tocontrol the vehicle in the autonomous driving mode and in order to slowthe vehicle down.
 19. The system of claim 18, wherein the one or moreprocessors are further configured to: for each given object of theplurality of objects, determine a priority level based on the confidencevalue and a type of the given object; compare the determined prioritylevels to a threshold value; and based on the comparison, determine thatthe first object allows for the braking level.
 20. The system of claim18, wherein the breaking level is a maximum braking limit for brakes ofthe vehicle.